Conversation
The issue said the conformance reader was the only thing missing, and that was true of the reader and not of the client. The pin was at 6ee7d80, which is before the engine had a Value::Bytes at all, so there was nothing for to_py to return and nothing for a parameter to become. The pin moves to 230581c and the three places that touch a value gain the arm they were missing. Reading gives bytes and binding takes bytes. Not bytearray and not memoryview: a value that came out of a result is a reading of what the file holds and nothing in Python should be able to write through it, and a parameter is read after the call that takes it returns, so a buffer the caller can still write through is a promise this client would be taking on trust. One type in all three directions, which is the one the loader already names for a byte string column. numpy gets an object array. A byte string column keeps the two buffers a string column keeps, so the walk is the string walk without the UTF-8 check, but the array cannot be an S one: S pads every cell to the longest and drops trailing nulls, which is a different value from the one stored. The reader turns hexits into octets itself rather than calling bytes.fromhex. The two agree on everything except a vertical tab, which fromhex drops and Rust's is_ascii_whitespace does not, and which whitespace fromhex drops has changed across the Python versions this client supports. A reader of the shared corpus that accepts a shade more than the reference one is a reader that lets a malformed case through on one client and not on another, which is the failure the encoding exists to make impossible. The pin bump also made ten words reserved that were not before: on, at, number, nothing, record, small, count, big, day and exact. Four test files used four of them as an alias or a column name and are renamed. Nothing in the client changed for that, only what the tests are allowed to call things. A byte string can be stored and cannot be read back yet, so the loader's refusal at src/load.rs:148 stays where it is. ColType has no byte string in it and the row walk has no arm that produces one, which is tamnd/zu#728. The corpus is 1399 cases, 1393 passed, 0 failed, 6 unsupported, and all six are a time written to the nanosecond, which is a digit finer than a Python datetime holds. The six test files this touches are 360 passed and 2 skipped, the two skips being the corpus tests that want ZU_CASES.
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #37.
The issue said the conformance reader was the only thing missing, and that was true of the reader and not of the client. The pin was at
6ee7d80, which is before the engine had aValue::Bytesat all, so there was nothing forto_pyto return and nothing for a parameter to become. The pin moves to230581cand the three places that touch a value gain the arm they were missing.Reading gives
bytesand binding takesbytes. Notbytearrayand notmemoryview: a value that came out of a result is a reading of what the file holds and nothing in Python should be able to write through it, and a parameter is read after the call that takes it returns, so a buffer the caller can still write through is a promise this client would be taking on trust. One type in all three directions, which is the one the loader already names for a byte string column.numpy gets an object array. A byte string column keeps the two buffers a string column keeps, so the walk is the string walk without the UTF-8 check, but the array cannot be an
Sone:Spads every cell to the longest and drops trailing nulls, which is a different value from the one stored.The reader turns hexits into octets itself rather than calling
bytes.fromhex. The two agree on everything except a vertical tab, whichfromhexdrops and Rust'sis_ascii_whitespacedoes not, and which whitespacefromhexdrops has changed across the Python versions this client supports. A reader of the shared corpus that accepts a shade more than the reference one is a reader that lets a malformed case through on one client and not on another, which is the failure the encoding exists to make impossible.The issue asked whether the comparison path keeps a byte string apart from the string that spells the same octets, since the case at
string.yamlline 995 asserts thatX'0041' = 'A'is not true and a reader that decoded octets into astrsomewhere would pass it for the wrong reason. It does, and there is now a test saying so on both sides of the comparison.The pin bump also made ten words reserved that were not before:
on,at,number,nothing,record,small,count,big,dayandexact. Four test files used four of them as an alias or a column name and are renamed. Nothing in the client changed for that, only what the tests are allowed to call things. Every other client will hit the same list on its next pin bump.A byte string can be stored and cannot be read back yet, so the loader's refusal at
src/load.rs:148stays where it is.ColTypehas no byte string in it and the row walk has no arm that produces one, which is tamnd/zu#728.What was run
Built and tested on a Linux box against CPython 3.12.3.
The corpus is 1399 cases, 1393 passed, 0 failed, 6 unsupported, and all six are a time written to the nanosecond, which is a digit finer than a Python datetime holds. Before this the six BYTES cases in
string.yamlwere refused at load rather than run.The six test files this touches are 360 passed and 2 skipped, the two skips being the corpus tests that want
ZU_CASES.ruff check,ruff formatandcargo fmtare clean.